Skip to content

Cache Anthropic capabilities and defer compaction lookup - #184

Merged
10-X-eng merged 3 commits into
10-X-eng:mainfrom
Anteriousis:fix/anthropic-capability-cache
Sep 9, 2026
Merged

Cache Anthropic capabilities and defer compaction lookup#184
10-X-eng merged 3 commits into
10-X-eng:mainfrom
Anteriousis:fix/anthropic-capability-cache

Conversation

@Anteriousis

@Anteriousis Anteriousis commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Reuse Anthropic model output capabilities across user turns through a parent-process cache (64 entries, five-minute TTL), scoped to model, endpoint and authentication identity. Child processes receive validated cached values and report successful lookups back to the parent. Credentials are hashed in cache identity and excluded from capability events.

A distinct compaction model is now resolved only when compaction is needed. Short successful turns avoid that lookup entirely; warm turns also avoid the primary lookup. Model-reported maximum output and older-SDK fallback behavior remain intact. Failed lookups and older-SDK fallback values are not cached.

Verification

  • Regression tests failed before implementation and pass afterward.
  • Exact test/build commands and results are listed below.

With local test dependencies on PYTHONPATH, $python denotes the bundled Python executable:

& $python -m pytest -q src/Mod/VibeCAD/vibecad_tests/test_provider_subprocess.py -k 'capabilities_survive or defers_unused' --tb=short
# Red: 7 failed, 52 deselected.
& $python -m pytest -q src/Mod/VibeCAD/vibecad_tests/test_provider_subprocess.py src/Mod/VibeCAD/vibecad_tests/test_gemini_provider.py --tb=short
# Green: 77 passed, 1 skipped (platform-specific).
& $python -m py_compile src/Mod/VibeCAD/VibeCADProvider.py
git diff --check

Tests exercise new provider instances, endpoint/model/auth changes, TTL expiry without sliding refresh, bounded size, failed/invalid metadata, lazy compaction and existing maximum-output/older-SDK behavior. Provider and CAD boundaries are mocked; no paid API calls or full FreeCAD GUI/C++ build were run.

Capabilities may remain stale for up to five minutes. Concurrent cold turns may each retrieve metadata; no cross-process shared disk cache or request coalescing is introduced.

Compatibility

  • Existing public functions/APIs remain present and compatible.
  • No preference keys, tool names or schema fields renamed/removed.
  • Output limits and existing generation defaults are preserved.
  • No deprecations or breaking public API changes.

Issues

Closes #178.

Before and After Images

Not applicable; no GUI changes.

@10-X-eng

10-X-eng commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Integration review against the performance main and PRs #180/#181/#183/#185: the existing combined suite passed 148 tests, but a new focused case exposes an interaction with #181.

When the primary generation returns max_tokens and a distinct compaction model's capabilities are first needed, model_max_tokens uses the same client whose max_retries #181 has already set to zero. Observed retry settings are [('primary', 2), ('compact', 0)], rather than preserving metadata retries for both lookups. The existing retry test covers only the eager primary lookup, and the lazy-compaction tests cover only unused compaction.

Hold this PR until deferred metadata lookup retains its intended transport retry budget without reintroducing nested retries on streamed generations. Quick regression: use a distinct compaction model, force a max_tokens response, fail the deferred metadata GET transiently, and assert successful recovery plus the existing six-attempt stream limit.

@10-X-eng 10-X-eng mentioned this pull request Sep 9, 2026
Merged
8 tasks
@10-X-eng
10-X-eng enabled auto-merge (squash) September 9, 2026 23:20
@10-X-eng

10-X-eng commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Resolved the #181 integration regression in 080c99d and merged current main into this branch without dropping either PR's tests. Deferred metadata temporarily restores the child-owned client's SDK retry allowance and restores the stream setting in a finally block.

Red: PYTHONPATH=src/Mod/VibeCAD /tmp/vibecad-pr-review-env/bin/python -m pytest -q src/Mod/VibeCAD/vibecad_tests/test_anthropic_retry_budget.py -k deferred_compaction --tb=short failed: only one compaction metadata GET was attempted.

Green: PYTHONPATH=src/Mod/VibeCAD /tmp/vibecad-pr-review-env/bin/python -m pytest -q src/Mod/VibeCAD/vibecad_tests/test_anthropic_retry_budget.py src/Mod/VibeCAD/vibecad_tests/test_provider_subprocess.py src/Mod/VibeCAD/vibecad_tests/test_gemini_provider.py --tb=short: 108 passed. Uses Anthropic 0.116.0 and mocked HTTP; no paid requests. The new test verifies recovery after two metadata 503 responses and max_retries=0 for both streamed generations. git diff --check passed.

Auto-merge enabled subject to required checks.

@10-X-eng
10-X-eng merged commit a442ec1 into 10-X-eng:main Sep 9, 2026
2 checks passed
@10-X-eng 10-X-eng mentioned this pull request Sep 9, 2026
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Cache Anthropic model capabilities across turns and defer unused compaction lookups

2 participants

@Anteriousis @10-X-eng